networkedlifeq21fandomcom-20200213-history
How ANN predicts Stock Market Indices? by Vidya Sagar Reddy Gopala, Deepak Ramu
SHORT ANSWER Stock market is a platform where investors buy and trade stocks or shares of companies. Investors can be small stock investors or institutional investors such as banks, insurance companies and mutual funds. Stocks or shares of companies are the investments to companies in smaller proportions for which investors get slice of ownership. Investors get paid profit in form of dividends, when companies are profitable, stock market investors make money through the dividends that the companies pay out and by selling the appreciated stocks yields profit called as capital gain. Stock market is the most preliminary way to raise money for the companies. By publicly selling stocks and shares, companies raise additional capital for expansion of their ventures. Advantage that the stock market offers is that the investors can easily exchange or trade their stocks anytime. This is an attractive feature compared to investments on immovable properties. Nowadays, significant portion of the funds related to saving and financing, flows to stock market investments instead of being routed to bank deposit and lending operations. Sources for such investment come from mutual funds, insurance investments, pension funds, etc. History of public investments date back to 13th century when venetian bankers started investing in government securities. In 16th century Italian companies were first to issue shares followed by companies in England. At Amsterdam exchange, the Dutch East India Company with continuous trade in stock was the first stock company to get a fixed capital stock. Since then many developed and developing countries have established and large markets. Increased business means rising stock prices and this rise is considered to be an up-coming economy. A country’s economy is indicated by strength of its stock market. Stock market or share market fall will effect a nation. First impact is that people will see fall in the value of their wealth. Reduction in pension value, reduces confidence of investor preventing the ability of the firm raising funds who raise funds by issuing more shares. Although people who decide to buy stocks are prepared to lose money. But losing money on the shares will make them hesitant to spend money on consumer products resulting in fall in consumer spending. This in-turn reduces profit that companies make and hits back at stock market and results in down-fall in economy of a country. We have had many examples of country’s economy getting affected due to the fall of stock market. One such great example is the “wall street crash of 1929” which was the most devastating stock market crash in the history of United States. This crash in the stock market resulted in a 10 year depression that affect all industries and industrialized countries. By 1920s U.S. stock market witnessed rapid expansion reaching its peak in August 1929 9. Excess production and rising stock market declined consumer demand. After reaching peaks, stock market witnessed steep fall due to which people sold their stocks and by 1932, stocks were worth of only 20% of their value when stock market reached its peaks. Stock market crash is considered as Great Depression, where half of the American banks failed and almost 30% of the America’s workforce were unemployed. This calls for the future insight of stock market. Wall Street Crash of 1929 is shown in the below figure . In past, there have been attempts made to predict stock market. Stock market prediction involves various factors and doesn’t have any mathematical relation with the indices of the stock market. Prediction of stock market comes as a big challenge. The various factors affecting rise or fall of stock market are the fundamental factors such as profit or dividends, margins of the share or stock market. Apart from the fundamental factors, there are various other factors that influence the stock prices. Other factors that influence stock market are the internal developments in companies, scandals, world events like natural calamities and civil unrest. Higher interest rates set by federal banks and reserve banks of countries might make the investors to sell or trade their stocks to avoid higher interest rates. Impact of exchange rates affects the business in foreign countries and thus affecting stock market. Although factors apart from fundamental factors can be considered as noise. But, it is difficult to ignore these factors while predicting the stock market. The act of determining the future value of stock of companies is called stock market prediction. Stock prediction can yield significant profit to the investor investing on stocks of different companies. Prediction is purely based on the previous stock values and it depends on different factors making prediction a tedious process. There have been attempts made to predict stock market at all levels using different analysis methods. Different analysis methods used to predict are fundamental analysis, technical analysis, time series analysis and machine learning techniques. Fundamental analysis is concerned with factors like stock prices and credibility of the company. A company’s past performance and credibility is evaluated and many performance ratios are created to analyze the stocks of the company. This analysis considers company operates well and hence requires capital to progress well. This type of analysis provide information such as financial statement analysis widely used by fund managers. It is a top-down analysis that includes different economies starting from global, country, sector and company level analysis. Technical analysis do not involve any of the fundamentals of a company. Future prediction basically depends on trends of the past price. Patterns are used to determine the future values and some of the patterns are candle stick patterns, head and shoulders and cup, saucer patterns etc. Time series analysis is similar to the technical analysis and it considers combination of market data along with public sentiment to predict stock market changes. Movements in data on particular days are used as key attributes such as supply and demand pattern for a set of previous days. Changes such as rapid increase is such as a positive attribute, rapid dip as negative attribute and changes which are negligible as neutral attribute. Machine learning analysis uses artificial intelligence to predict stock market. Usually artificial neural network is used as human brain, fed with massive data to interpret stock market. Since there is no mathematical relation between inputs and outputs of the predicted values, artificial neural networks determines the hidden patterns and unknown parameters involved in prediction. Artificial neural network will predict stock market using the past experience that the system has learnt. For the artificial neural network to predict, system has to learn the effects of the various factors affecting the indices of the data to be predicted. For the system to predict, system undergoes two process, one being learning process and learning process is used to predict. During learning process, data that of previous days are fed on to the system and the determined output is compared with the predicted output for a particular set of input values. The error in the predicted value is used to correct the system and thus helping the system reduce errors in predicted values. The prime component of an artificial neural network is a neuron similar to the neuron in the human brain. Neuron in artificial neural network is considered as the mathematical function. Artificial neural network consists of several neurons connected to each other. There is three layer of neurons used in the prediction of stock market, those are the input layer, output layer and the hidden layer where the computation takes place for the prediction. Number of neurons in the input layer is decided by the number of inputs being provided for the prediction. Connection between neurons has weights associated with it to signify the strength of connection between neurons. Errors calculated after prediction in learning phase is used to update weights so that neurons identify the inputs data those are significant and insignificant. Artificial neural network does not make use of insignificant data for prediction once the system is ready for prediction after learning phase. LONG ANSWER Artificial neurons As we see in biological neural network or the structure, the transmission of the signal from one neuron to another takes place through synapses which is a chemical process in which specific substances are released from the sending side of the junction to receiving side. The concept of artificial network was developed based on such transfer process. The biological neurons are the inspiration for designing artificial neural network. The electric signal is raised or lowered inside the receiving cell or junction. There is a threshold value which decides the raise and lowering of signal at receiver end. If the graded potential is higher than the threshold value, then the neutron fires. The below figure is the widely used example block for a single neuron. Based on the requirements, slight modifications are done for neuron structures. The structure of the artificial neuron given in above figure has N inputs which are denoted as u''1, ''u''2…''uN and one output which is denoted by x. ''Each input to the neuron has a weight which are denoted by ''w1, w2…wN. The threshold in the artificial neuron is generally represented by ��. The activation factor for each neuron is represented by ‘��’. The formula for calculating activation factor is given by the below formula. a=\frac{1}{N}\sum_{j=1}^N\left(w_{j}u_{j}+\theta\right) The inputs and the weights are generally real values. A negative value for weight indicates inhibitory connection and the positive connection represents excitatory connections. For simplicity, we can combine the threshold value with the summation part. Let us assume that the input u0 = +1. And the weight w0 = ��. Hence the activation formula is given by, a =\frac{1}{N}\sum_{j=0}^N\left(w_{j}u_{j}\right) The output value of the artificial neuron is a function of its activation function. Thus, x'' can be given by the below formula, x=f\left(a\right) '''Artificial neural network' In machine learning and cognitive science, artificial neural networks (ANNs) are biological neural networks which are used to approximate functions which depends on large number of inputs and provide an output. Artificial neural networks are represented as systems of interconnected or clustered "neurons" which exchange signals and messages between each other. The connections have weights which can be tuned based on experience, making neural adaptive to inputs and capable of learning based on output. A single neuron cannot be used to develop or design a complete Boolean function, thus we use a cluster of neurons. The cluster of neurons together is called as artificial neural network. By using this cluster of neurons we can design and develop huge Boolean functions. In such a cluster of neurons, we provide each neuron with particular indices to identify them. We don’t have to understand the system or the network, as it trains itself. This is one of the major advantage of ANN 1. Another advantage of the network is it will learn ignoring the data which does not improve the output of the system in the training phase 2. Thus to activate ith neuron, the formula shown below can be used, a_{i}=\sum_{j=1}^N\left(w_{ij}x_{j}+\theta_{j}\right) Where xj represents the output of another neuron or an external input for the network. There is a training phase where parameters called as weights are found from the sample used for training and back propagation algorithm is used for this training of network. These updated weights are used in prediction phase using the same equation formed by the network developed in training period. There are many types of artificial neural networks based on the user requirements. Here, we are considering feedforward neural network which is shown in the below figure. The neurons in the network forms many layers between inputs and output. Neurons in the network get the data from the input signals or from the previous nodes or neurons. The last layer is called as output layer and the first layer is called as input layer. And the in between layers are called as hidden layers or middle layers. If there is one single hidden layer, then it’s called as single layer network. Else if there are many layers in the hidden layers, then it’s called as multilayer networks. In this particular project, we are considering the feedforward network with back propagation algorithm. Back propagation learning algorithm is discussed below. Back propagation learning algorithm The one of the biggest improvement to the ANN network was “Back propagation algorithm”. Back propagation algorithm 4, and 5 mainly improved the speed of training the network. Basically the process of back propagation is to propagate the errors from the output layers to the input layers. Back propagation is very much needed as the network do not have a fixed target to achieve, thus they train the network based on the back propagated error. As the error is back propagated the weights are updated. Training is continued until the errors obtained are small enough. The main steps used in back propagation algorithm are as follows. 1. Normalized data samples are fed and corresponding output value is found. 2. Compute the error value between the target value and the estimated value. 3. The connections are updated and the connections are adjusted. 4. IF error > Tolerance THEN go to step 1 ELSE stop. Generally back propagation falls into a general category of gradient descent algorithms, which intends to find the maxima or minima of a function by iteratively sliding the direction of negative of slope the function to be maximized or minimized. The main objective of back propagation is to minimize error function. Thus, the error functioned to be can be given by \xi_{av}=\frac{1}{N}\sum_{n=1}^N\xi(n) In this particular algorithm, the connection weights are updated pattern by pattern in each epoch. The weights are adjusted with respect to the errors computed in each epoch in the network. The average of all these individual weights are an estimate of true change that result from the modification of weights based on the error function. The gradient descent strategy is used to reduce the error. The chain rule for differentiation is \frac{\partial \xi(n)}{\partial w_{ij}(n)}=\frac{\partial \xi(n)}{\partial e_{j}(n)} \frac{\partial e_{j}(n)}{\partial y_{j}(n)}\frac{\partial y_{j}(n)}{\partial v_{j}(n)} \frac{\partial v_{j}(n)}{\partial w_{ij}(n)} On simplification, \frac{\partial \xi(n)}{\partial w_{ij}(n)}= -e_{j}(n)\phi_{j}\backprime((v_{j}(n))y_{i}(n) Thus, the final equation for updating becomes \triangle w_{ij}(n)= \eta\delta_{j}(n)y_{i}(n) Where, \delta_{j}(n)=-\frac{\partial \xi(n)}{\partial v_{j}(n)} =\frac{\partial \xi(n)}{\partial e_{j}(n)} \frac{\partial e_{j}(n)}{\partial y_{j}(n)}\frac{\partial y_{j}(n)}{\partial v_{j}(n)} = e_{j}(n)\phi_{j}\backprime((v_{j}(n)) For the last layer (output layer) and the hidden layers, \delta_{j}(n) =\phi_{j}\backprime((v_{j}(n))\sum_k\delta_{k}(n)w_{kj}(n) Batch learning scheme is used for updating the weight, all the training samples of data are fed into network in each epoch. The change in weight is computed from each and every sample of data. For each epoch, all the training data samples are inputted. For practical implementation, ANNs have simple structure with less number of hidden layers as number of layers exponentially increases the time taken for training the system. Thus, by giving the set of samples for training and testing the network can be used for prediction. The number of input and output layers depends on the input and output features. There are few disadvantages of back propagation learning as well, they are as follows. * The convergence of back propagation learning is very slow. * The convergence obtained from back propagation is not guaranteed. * The convergence may happen at any local minimum on the error surface. * Input scaling and normalization is required for learning. * Activation function used by neurons are to be differentiable. Artificial Neural Network features Activation function: '''The user can activate neuron using three activation functions. The activation functions used are common for all the neurons in the network. They are as follows. * ''Unipolar sigmoid function: f(x)= \frac{1}{1+e^{-\lambda x}} '' * ''Bipolar sigmoid function: f(x)= \frac{2}{1+e^{-\lambda x}}-1 '' * ''Tan hyperbolic function: f(x)= \frac{e^{\lambda x}-e^{-\lambda x}}{e^{\lambda x}+e^{-\lambda x}} '' * ''Radial basis function: f(x)= \frac{1}{\sqrt{2 \pi\sigma}} e^ /2} '' '''Hidden Layers and Nodes: '''The network which we train in order predict the stock market have arbitrary number of hidden layers and arbitrary number of nodes in each layer. Both are decided by the user to increase the performance of the network. '''Data Normalization: '''The data which is fed to the ANN is initially normalized. The input sample vector are normalized in order to gain unit variance and zero-mean features. The target values are normalized such that the functions are bounded between the limits. The below table shows the minimum and maximum values of the activation function. The test data set is scaled to the same extent as data used for training. The output value for the test sample is scaled back to the same factor as training sample set. '''Stopping Criterion: '''The learning rate ''n ''is used to control the rate of convergence for this back propagation algorithm. Larger the value of ''n ''faster the convergence and vice versa. The stopping criteria is required for an algorithm to ensure that it does not run forever. The back propagation stops if any of the following conditions are met: * Change in error falls below the threshold which is set by user from one epoch to another. * When the error value begins to increase. There is a certain minimal relaxation factor, which to be maintained by the network. * If the number of iterations goes beyond certain limit. The limit will be set by the user. '''Error Calculation: '''The error for convergence is calculated as the RMS value of error between estimated value and actual value (output). The same error is used to report the performance of algorithm on the test sample. '''Cross-validation: '''The error is calculated using the training set, which is then used in estimating the output of testing set. Previous to testing, we can check the output by testing it on another sample set which is called as Cross-validation set. '''Model working and analysis The overall prediction happens in two stages, they are training phase and prediction phase namely. Two different sets of data samples are used in both the phases. In this part, we have explained briefly about both the phases and analyzed it’s working. Training Phase: The training phased by itself can be divided into two stages. They are propagation phase and weight update phase. * Propagation phase: In the propagation phase firstly the normalized inputs are feds into the network. The formula for normalizing the data is shown below, where,⦁ V'=v'=\frac{v-minA}{maxA-minA} (new_{maxA}- new_{minA})+ new_{minA} V’ = Normalized input and V = Actual input. minA and maxA boundary values of the old data range. newminA and newmaxA boundary values of the new data range. In this case it is -1 to 1 6. In the propagation phase, the input data set is taken in at each epoch. And based on the input values, and the pre-known output the ANN develops a network in such a way that all the neurons contribute to in obtaining the predetermined output. Thus, the error is calculated based on the formula given below.⦁ e=\frac{actual output -ANNoutput}{actual output} The error ‘''e''’ which is generated in propagation phase is used to update weight in weight updating phase. * Weight updating phase: '''As seen in the ANN block diagram above, the weights are multiplied with the input data set to produce the required output. The weights are calculated using the formula shown below. '''Updated weight = weight(old) + learning rate * output error * output(neurons i) * output(neurons i+1) * (1 – output(neurons i+1)) The above formula is used for calculating and updating the weights in every epoch. Both the propagation phase and weight updating phase are repeated in every epoch until the sum of square error is zero or close to zero. Prediction Phase: The neural network is trained completely until the error rate is very low or zero, then the trained network is used to predict the output value in prediction phase. The normalization and scaling factors remains the same for prediction set of data too. In other words, the input data set is scaled and normalized to the same factor as the training set of data. The trained system firstly validates its working using cross-validation set of data sample. Data pre-processing In our project for predicting the stock market, we use Nifty Sensex data. Generally information that is embedded in Nifty Sensex data are: o The date o Time of day o Opening price o Closing price o High stock value o Low stock value o Fractional change in price from previous time step. Out of these attributes, the inputs that make sense to our prediction network are opening, closing, high and low values. The output consists of single attribute that is closing price of the stock market. The data was further divided into different sets of data. That is 60% for training the ANN, 40% for testing the system. The 60% of training set is further divided into 40% for training and 20% for cross-validation of the model. Few samples of data are taken together to feed the system. A sliding window is used to combine the data points as follows: On = (On-k+1, On-k+2… On) Here k is the number of previous observations used to predict the next value. The target or estimated values are the next time step: Tn = On+1 For normalizing the input value, MatLab functions such as ‘zscore’ can be used. It subtracts the mean from the sample and the result is divided by the variance of the data. The target outputs were also normalized based on the factor which was used for normalizing input values. The normalization is done such that it is bounded between upper and lower limit which differs from one activation factor to another. The limits are nothing but the maximum and minimum values which are shown in table. Stock Indices prediction The stock market indices are predicted using the system which was developed as shown in ANN figure. The variables decided by user are used to set up the system as required. Let us consider the experimental results of the analysis done by Abhishek Kar 7. The variables which are decided by the author are as given below: * Inputs (Open, Low, High, Close) * Outputs (Close value) * Percentage of training data (60%) [ 40% training + 20% Cross validation ] * Percentage of testing data (40%) * Number of previous data points considered for training (5) * Learning rate, η (0.001) * Number of hidden layers, nH (2) * Number of nodes in each hidden layer, n (100,50) * Maximum number of epochs (30) * Activation function (Unipolar Sigmoid) * Value of the activation function parameter (0.5) Thus using the above parameters, the system is developed. The RMS error is plotted against epochs in the below figure. The graph below clearly shows that RMS error gradually decreases from one epoch to another as the system gets more accurate. The various parameters which can be varied by the user were varied to find the ideal scenario where the system predicted the stock market indices with more accuracy. Thus, maximum accuracy and efficiency was found. Below three graphs shows the closing value of stock prices predicted using single, two and three hidden layers. Then, the analysis was for single hidden layer, and varying number of nodes in hidden layer. Thus the graph is plotted for errors against varying number of nodes. The below figure, shows the training and testing error plotted against number of nodes. Then a graph is plotted where training and testing error were plotted against varied double hidden layered neural network. The number of previous data sets used for testing the data was varied for the double hidden layered neural network. The graph plotted is as shown in below figure. Stock Prediction results The results obtained in both the cases were approximately accurate. As seen from above figures the prediction if fairly accurate unless there are huge variations in actual data like in right extreme of the figure, where it becomes impossible to exactly predict the change as the change is unexpected and huge. The minimum error over the testing and training data was as low as 3.5% for single layer situation. The best seen accuracy was achieved in the default case of 96%. And the average accuracy over all the cases was 88%. Thus we see that artificial neural networks are effective tool for predicting stock market indices. Artificial neural network not only predicts Stock market indices, but can also be used for weather forecast, indoor positioning systems and many other places where there is no rigid formulas or equations to predict the next sample in the sliding window. References 1 “''Feedforward neural Networks: An Introduction''” by Simon Haykin page (2-4) 2 “''Artificial Intelligence a modern approach''” (second edition) by Stuart Russell, Peter Norvig 2004 3 Rumelhart, D.D.m Hinton, G.E. and Williams, R.J., Learning Internal Representation, Man, and Cybernetics (SMC‟91), 1991. 1913-1918. 4 Simon Haykin, “''Neural Network A Comprehensive Foundation''”, second edition, Prentice Hall, 1998, page 161 – 173. 5 Y.-Q. Zhang and A. Kandel, “''Compensatory Genetic Fuzzy Neural Networks and Their Applications'',” Series in Machine Perception Artificial Intelligence, Volume 30, World Scientific, 1998. 6 VanEyden, R. J. (1996). The application of neural networks in the forecasting of share prices. Finance & Technology. 7 Stock Prediction using Artificial Neural Networks 8 Price Prediction of Share Market using Artificial Neural Network (ANN) 9 http://www.investopedia.com/articles/07/stock-exchange-history.asp 10 http://www.history.com/topics/1929-stock-market-crash